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PREFACE 


The  United  States  Air  Force  Academy  is  a  relatively  young 
institution.  Although  just  ever  thirty  years  old,  it  has 
produced  nearly  20,000  graduates.  For  a  number  of  reasons, 
very  little  research  on  the  performance  of  these  graduates  in 
the  Air  Force  has  been  accomplished. 

In  the  last  few  years,  however,  the  Academy  has  been 
initiating  a  major  research  effort  to  investigate  the  perfor¬ 
mance  of  its  graduates  on  active  duty.  It  hopes  to  be  able  to 
study  the  relationship  of  Academy  training  programs  and  admis¬ 
sions  criteria  to  graduate  performance  in  the  Air  Force.  Such 
feedback,  whether  positive  or  negative,  could  be  a  valuable 
source  of  ideas  for  modifying  Academy  admissions  and  training 
programs  to  better  accomplish  the  mission  of  producing  out¬ 
standing  Air  Force  officers. 

The  Academy  is  a  major  source  of  pilot  candidates  for  the 
Air  Force.  Over  half  of  each  graduating  class  enters  pilot 
training  programs.  Prior  to  graduation,  these  cadets  are 
screened  for  flying  training  in  the  T-41  Pilot  Indoctrination 
Program  at  the  Academy.  Cadets  must  successfully  complete  this 
program  in  order  to  enter  into  any  Air  Force  pilot  training 
after  graduation  and  commissioning. 

To  assist  with  the  overall  graduate  research  effort  now 
underway,  this  project  investigates  the  relationships  between 
performance  in  the  T-4^.  PIP  at  the  Academy  and  graduate  perfor¬ 
mance  in  pilot  training.  It  is  hoped  that  the  findings  of  this 
study  will  shed  some  light  on  these  relationships  and  open  the 
door  to  future  investigators  for  further  research  in  specific 
areas  of  interest. 
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EXECUTIVE  SUMMARY 


Part  of  our  College  mieeion  is  distribution  of  the 
students*  problem  solving  products  to  DoD 
sponsors  and  other  interested  agencies  to 
enhance  insight  into  contemporary,  defense 
related  issues.  While  the  College  has  accepted  this 
product  as  meeting  academic  requirements  for 
graduation,  the  views  and  opinions  expressed  or 
implied  are  solely  those  of  the  author  and  should 
not  be  construed  as  carrying  official  sanction. 


REPORT  NUMBER  87-2020 
AUTHOR(S)  major  alison  l.  piotter,  usaf 

TITLE  PERFORMANCE  COMPARISON:  USAF  ACADEMY  T-41  PILOT 

INDOCTRINATION  PROGRAM  VERSUS  USAF  PILOT  TRAINING 


I.  Purpose:  To  investigate  the  relationship  between  cadet 
performance  in  the  USAF  Academy  (USAFA)  T-41  Pilot  Indoctrina¬ 
tion  Program  (PIP)  and  graduate  performance  in  two  USAF  pilot 
training  programs:  Undergraduate  Pilot  Training  (UPT)  and 
Euro-NATO  Joint  Jet  Pilot  Training  (ENJJPT) . 

II.  Problem:  USAFA  is  a  major  source  of  pilot  candidates  for 
the  Air  Force.  Over  half  of  each  class  enters  pilot  training 
programs  after  graduation.  Cadets  must  successfully  complete 
the  T-41  PIP  in  order  to  enter  any  Air  Force  pilot  training 
program  as  officers.  USAFA  is  currently  interested  in  investi¬ 
gating  the  relationship  between  performance  in  T-41  PIP  and 
UPT/ENJJPT.  This  study  will  complement  the  recent  research 
initiatives  underway  at  USAFA  to  study  '.‘he  relationship  of 
USAFA  training  and  admissions  programs  to  graduate  performance 
in  the  Air  Force. 

III.  Data :  USAFA  currently  has  no  established  program  for  the 
permanent  collection,  retention,  or  analysis  of  cadet  perfor¬ 
mance  data  in  T-41  PIP.  However,  ad  hoc  data  for  the  classes 
of  1984  and  1985  is  available.  Only  two  performance  measures 
reflecting  overall  cadet  performance  are  available:  the  T-41 
Checkride  Score  and  the  Prediction  for  UPT.  HQ  ATC  retains  a 
certain  amount  of  performance  data  on  each  UPT  graduate  in  its 
data  base.  The  factors  that  represent  overall  UPT  performance 
are  the  Category  Checkride  Composite  score,  the  ATRB  recom- 


CONTINUED": 


mendation,  the  Distinguished  Graduate  (DG)  award,  and  tne  ATC 
Commander’s  Trophy  award.  A  comparison  of  the  T-41  PIP  and  UPT 
performance  data  indicates  that  there  is  a  direct  relationship 
between  the  T-41  Checkride  Score  and  UPT  performance,  and  a 
similar  direct  relationship  between  the  T-41  Prediction  for  UPT 
score  and  UPT  performance.  Tests  of  significance  and  correla¬ 
tion  between  the  data  are  less  conclusive.  While  the  testa  for 
significance  show  a  statistically  significant  relationship 
between  the  T-41  PIP  and  UPT  performance,  the  tests  for  corre¬ 
lation  do  not  show  the  relationship  to  be  a  strong  one.  How¬ 
ever,  the  remainder  of  the  analysis  performed  in  this  study 
suggests  that  strong  relationships  exist  between  the  T-41  PIP 
and  pilot  training  performance  measures. 

IV.  Findings :  Performance  in  T-41  PIP  is  closely  and  directly 
related  to  performance  in  UPT/ENJJPT  in  the  sample  studied. 

V.  Recommendations :  USAFA  should  continue  to  assess  cadet 
performance  in  T-41  PIP  using  the  performance  measures  studied. 
USAFA  should  also  establish  an  ongoing  program  for  tracking  and 
analyzing  T-41  PIP  performance  with  respect  to  subsequent  per¬ 
formance  in  USAF  pilot  training  programs. 
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Chapter  One 


INTRODUCTION 

PURPOSE 


The  USAF  Academy  (USAFA)  has  recently  established  a  com¬ 
prehensive  evaluation  program  of  its  graduates.  The  purpose 
of  the  program  is  to  gather  data  on  the  performance  and  capa¬ 
bilities  of  recent  graduates  and  relate  the  findings  to  USAFA 
admissions  criteria  and  training  programs  (3:21). 

This  research  project  is  sponsored  by  USAFA  and  supports 
the  overall  graduate  evaluation  program.  It  will  investigate 
one  specific  portion  of  that  program,  the  relationship  between 
cadet  performance  in  the  T-41  Pilot  Indoctrination  Program 
(PIP)  at  USAFA  and  graduate  performance  in  two  Air  Force  pilot 
training  programs:  Undergraduate  Pilot  Training  (UPT)  and 
Euro-NATO  Joint  Jet  Pilot  Training  (ENJJPT) . 


BACKGROUND 

The  T-41  PIP  at  USAFA  is  a  required  course  for  all  physi¬ 
cally  qualified  first  class  (senior  year)  cadets  who  volunteer 
to  attend  the  UPT  or  ENJJPT  programs .  The  goals  of  this  pro¬ 
gram  are  to  identify  those  cadets  who  have  the  basic  aptitude 
to  become  Air  Force  pilots,  motivate  them  toward  a  rated  career 
in  the  Air  Force,  and  minimize  attrition  of  USAFA  graduates  in 
pilot  training  (9:37).  Elimination  for  flying  deficiency  from 
the  T-41  PIP  will  preclude  a  cadet  from  entering  any  Air  Force 
pilot  training  program  after  graduation  from  USAFA  (3:34). 

The  557th  Flying  Training  Squadron  conducts  the  T-41  PIP  at 
the  USAFA  airfield  and  uses  Air  Force  pilots  as  instructors. 

The  course  of  instruction  includes  ground  training  classes  in 
airmanship,  safety,  and  aircraft  systems,  as  well  as  a  flying 
phase  consisting  of  15  sorties  (21  hours  of  flying  time) .  When 
a  cadet  becomes  proficient  in  all  required  flight  maneuvers  and 
is  cleared  for  solo  flight,  he  or  she  must  successfully 
complete  a  final  checkride  in  order  to  finish  the  course 
(9:37)  . 
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UPT  is  conducted  by  the  Air  Force  at  five  bases  in  the 
continental  United  States.  The  goal  of  UPT  is  to  qualify 
officers  for  the  aeronautical  rating  of  pilot  and  to  prepare 
them  for  future  responsibilities  as  military  officers  and 
leaders.  The  program  includes  flying  training  to  teach  the 
principles  t.id  techniques  of  operating  high  speed  jet  aircraft, 
ground  training  to  supplement  and  reinforce  the  flying  train¬ 
ing,  and  general  officer  development  training.  The  course  is 
49  weeks  in  duration  and  includes  approximately  185  total  hours 
of  flying  time  in  the  T-37  and  T-38  aircraft  (7:1). 

ENJJPT  is  designed  to  qualify  students  from  the  United 
States  and  11  Euro-NATO  countries  as  pilots  capable  of  flying 
high  speed  jet  fighter-type  aircraft  (12:16,30).  It  is  con¬ 
ducted  by  the  USAF  at  Sheppard  AFB ,  Texas  and,  like  UPT,  in¬ 
cludes  flying,  ground,  and  officer  development  training. 
Slightly  longer  in  duration  than  UPT,  the  course  lasts  55  weeks 
and  includes  260  total  flying  hours  in  the  T-37  and  T-38 
aircraft  (12:13) . 


NEED 

There  is  a  growing  interest  in  the  area  of  predicting 
successful  performance  in  pilot  training.  At  the  request  of 
Air  Training  Command,  the  Air  Force  Human  Resources  Laboratory 
(AFHRL)  has  conducted  research  since  1979  to  improve  the 
screening  process  of  selecting  and  classifying  candidates  for 
pilot  training  (4:258-259).  In  addition,  the  AFHRL  has 
recently  completed  research  investigating  the  value  of  the  USAF 
Flight  Screening  Program  (FSP)  in  reducing  pilot  training 
attrition  (15:i) .  Similar  to  the  T-41  PIP  at  USAFA,  the  USAF 
FSP  involves  14  hours  of  T-41  flying  for  pilot  training  candi¬ 
dates  commissioned  through  sources  other  than  USAFA  (15:1). 

USAFA  has  also  recently  become  interested  in  establishing  an 
ongoing  tracking  program  to  compare  the  T-41  PIP  performance  of 
its  cadets  with  their  subsequent  pilot  training  performance 
(16:  — ).  This  research  project  is  designed  to  assist  that 
effort  by  providing  an  initial  overview  of  the  T-41  PIP  and 
pilot  training  performance  relationships.  Also,  the  USAFA 
Directorate  of  Graduate  Evaluation  intends  that  this  project  be 
used  as  the  foundation  for  the  development  of  further  research 
to  support  this  proposed  tracking  program  (16: — ). 
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OBJECTIVES 


This  study  will  accomplish  the  following  objectives: 

-  Determine  appropriate  T-41  PIP  and  pilot  training 
performance  measures  to  be  used  for  comparison 

-  Evaluate  the  statistical  relationships  between  the 
T-41  PIP  and  pilot  training  performance  measures 

-  Summarize  findings  and  recommendations  concerning 
the  relationships  between  the  performance  measures 


SCOPE 


This  study  will  examine  the  flying  performance  of  graduates 
of  the  USAFA  classes  of  1984  and  1985.  These  two  classes 
constitute  the  only  sample  for  which  both  T-41  PIP  and  Air 
Force  pilot  training  performance  data  were  available.  The 
investigation  will  focus  on  these  graduates  primarily  as  a 
combined  group,  although  the  performance  of  each  class  will 
also  be  reported  separately  in  most  areas  of  investigation. 

The  intent  of  the  project  is  to  study  the  performance  of  USAFA 
graduates  as  a  whole,  and  not  to  focus  extensively  on  perfor¬ 
mance  differences  that  may  occur  between  the  classes. 

The  examination  of  pilot  training  performance  will  be 
limited  to  the  UPT  and  ENJJPT  programs  only.  Although  there  is 
a  third  pilot  training  program  to  which  USAFA  graduates  may  be 
assigned.  Undergraduate  Helicopter  Training  (UHT) ,  no  per¬ 
formance  data  was  available  for  this  program.  In  addition,  UHT 
accounts  for  only  approximately  five  percent  of  all  USAFA 
graduates  entering  pilot  training  programs  (18: — ).  Therefore, 
the  term  "pilot  training"  as  used  in  this  study  will  refer  to 
the  UPT  and  ENJJPT  pilot  training  programs  only. 


OVERVIEW 

In  order  to  compare  performance  in  the  T-41  PIP  and  Air 
Force  pilot  training  programs,  appropriate  performance  measures 
for  each  program  must  be  identified.  An  analysis  and  selection 
of  available  performance  measures  for  T-41  PIP  is  presented  in 
Chapter  Two.  Similarly,  Chapter  Three  analyzes  and  selects 
available  measures  of  performance  for  the  pilot  training  pro¬ 
grams  . 

Once  the  performance  measures  to  be  used  for  comparison 
have  been  identified,  the  relationships  between  performance  in 
these  programs  can  be  examined.  Chapter  Four  presents  the 
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analysis  of  these  performance  relationships.  Xt  describes  the 
data  characteristics  of  the  performance  measures,  selects 
appropriate  analysis  techniques  based  on  those  characteristics, 
&nd  reviews  the  results  of  the  data  analysis. 

On  the  basis  of  this  analysis,  Chapter  Five  presents  the 
overall  findings  of  this  study  regarding  the  relationships  be¬ 
tween  performance  in  T-41  PIP  and  pilot  training.  In  addition, 
it  makes  recommendations  for  further  action  in  the  study  of 
these  relationships. 
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Chapter  Two 


T-41  PIP  PERFORMANCE  MEASURES 

Detailed  student  performance  data  in  T-41  PIP  is  collected 
and  maintained  throughout  the  training  program  (ll:Ch  7) . 
However,  USAFA  currently  does  not  have  a  regular  program  for 
collecting  or  maintaining  T-41  PIP  student  performance  data  on 
a  long-term  basis.  The  limited  T-41  student  performance  data 
used  in  this  study  was  collected  by  USAFA  on  a  one-time  basis 
in  the  summer  of  1985  in  anticipation  of  further  research  in 
this  area  (16: — ) . 

In  the  data  collected,  there  were  only  two  performance 
measures  available  that  reflected  overall  student  performance 
in  T-41  PIP.  This  chapter  describes  these  performance  measures 
and  presents  the  rationale  for  their  use  in  this  study. 


T-41  CHECKRIDE  SCORE 

The  T-41  PIP  at  USAFA  is  graded  on  a  pass/fail  basis. 

During  the  ground  training  phase  of  instruction,  cadet  perfor¬ 
mance  is  measured  by  emergency  procedures  quizzes,  written 
standardization  quizzes,  and  an  academic  exam,  each  of  which 
has  a  minimum  passing  percentage.  In  the  flying  phase  of 
training,  student  progress  is  monitored  on  each  practice  sortie 
by  awarding  a  subjective  grade  for  each  subarea  or  maneuver 
evaluated,  as  well  as  an  overall  grade  for  the  mission  (21: — ). 
These  subjective  grades  are  based  on  the  judgment  of  the 
instructor  pilot  (IP)  and  are  awarded  according  to  the 
following  criteria  (10:2): 

-  Unable  to  Accomplish  (U) .  The  student  is  unsafe  or  lacks 
sufficient  knowledge,  skill,  or  ability  to  perform  the 
operation,  maneuver,  or  task. 

-  Fair  (F) .  The  student  performs  the  operation,  maneuver, 
or  task  safely,  but  has  limited  proficiency.  Deviations 
occur  which  detract  from  performance. 

-  Good  (G) .  The  student  performs  the  operation,  maneuver, 
or  task  satisfactorily.  Deviations  occur  but  are 
recognized  and/or  corrected  in  a  timely  manner. 
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-  Excellent  (E)  .  The  student  perforins  the  operation, 

maneuver,  or  task  correctly,  efficiently,  and  skillfully. 
Minor  deviations  occur  but  do  not  detract  from  overall 
performance. 

The  checkride  at  the  end  of  the  T-41  course  is  graded  in  a 
similar  manner.  Twenty-nine  maneuvers  are  evaluated  using  the 
U/F/G/E  grading  system  outlined  above  (14:Atch  3).  If  any 
maneuver  is  graded  "U"  (Unable  to  Accomplish) ,  the  subjective 
overall  grade  for  the  checkride  is  Unsatisfactory  (10:9). 
Otherwise,  the  IP  awards  an  overall  grade  of  Fair,  Good,  or 
Excellent,  based  on  a  subjective  evaluation  of  the  student's 
overall  performance.  A  cadet  must  receive  a  subjective  overall 
checkride  grade  of  Fair,  Good,  or  Excellent  in  order  to 
successfully  pass  the  T-41  program  (10:9;  21: — ). 

Following  the  final  checkride,  each  cadet's  checkride  per¬ 
formance  is  converted  into  a  quantitative  score  expressed  as  a 
percentage.  In  each  maneuver  area,  the  U/F/G/E  grade  is 
awarded  a  certain  number  of  points.  The  point  values  for  each 
grade  will  vary  based  on  the  relative  weight  of  that  maneuver 
area.  The  overall  percentage  is  computed  by  adding  up  all  the 
maneuver  area  point  values  awarded  and  dividing  the  sum  by  the 
maximum  possible  points  (21: — ). 

This  overall  percentage  grade  will  be  referred  to  in  this 
study  as  the  "T-41  Checkride  Score."  This  score  does  not 
affect  the  student's  overall  final  course  grade,  which  is 
pass/fail.  Instead,  it  is  used  by  USAFA  primarily  as  a  quality 
control  device  to  compare  the  consistency  of  each  IP’s  overall 
subjective  grading  on  the  final  checkrides  with  that  of  other 
instructors.  This  is  done  by  comparing  the  overall  subjective 
grades  awarded  by  the  IP  with  the  computed  percentage  grades  on 
those  same  checkrides.  This  process  helps  to  identify  IPs 
whose  overall  subjective  ratings  may  be  higher  or  lower  than 
the  norm  as  compared  to  the  computed  checkride  percentage 
scores  (21 :  — )  . 


POTENTIAL  FOR  UPT 


Another  measure  used  to  evaluate  cadet  performance  in  the 
T-41  PIP  is  the  Potential  for  UPT  rating.  This  rating  is  used 
primarily  by  supervisors  as  a  tool  in  determining  which  cadets 
should  be  considered  for  the  ENJJPT  program.  The  cadet's 
flight  commander  is  responsible  for  assigning  this  grade  and 
uses  inputs  from  the  cadet's  IP.  Factors  considered  in  this 
subjective  grade  include,  but  are  not  limited  to,  daily  flying 
performance,  standardization  and  academic  quiz  performance,  and 
checkride  results.  This  area  is  graded  according  to  the 
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following  criteria  (14:Atch  2): 

Rating  Criteria 

1  Student  has  marginal  aptitude  and  marginal 

academic  scores.  Limited  potential  to 
finish  UPT. 

2  Student  has  slightly  below  average  flying 
aptitude  and  academic  ability.  May  have 
difficulty  completing  UPT. 

3  Student  has  average  flying  aptitude  and 
academic  ability.  Should  complete  UPT  with 
no  difficulty. 

4  Student  has  slightly  above  average  flying 

aptitude  and  academic  ability.  Should 
complete  UPT  in  the  top  50  percent  of  the 
class . 

5  Student  has  exceptional  flying  aptitude  and 
academic  ability.  Should  finish  in  the  top 
25  percent  of  the  class.  Should  be  consi¬ 
dered  for  ENJJPT. 

Because  the  ENJJPT  program  is  currently  not  open  to  female 
USAF  students,  female  cadets  are  not  considered  for  ENJJPT, 
although  they  may  be  awarded  a  "5"  Potential  for  UPT  rating 
(21s—)  . 


PERFORMANCE  MEASURES  SELECTED 


The  T-41  Checkride  Score  and  the  Prediction  for  UPT  will 
both  be  used  in  this  study  as  measures  of  performance  in  T-41 
PIP.  Both  scores  are  useful  in  differentiating  between  cadets 
in  T-41  performance,  as  well  as  in  comparing  each  cadet's  T-41 
performance  with  his  or  her  subsequent  performance  in  pilot 
training. 

In  the  discussion  of  the  T-41  Checkride  Score  presented 
earlier  in  this  chapter,  mention  was  made  of  a  subjective 
overall  checkride  grade  (U/F/G/E)  awarded  by  the  IP.  This 
grade  should  not  be  confused  with  the  T-41  Checkride  Score, 
which  is  expressed  as  a  percentage  of  total  points  possible  on 
the  checkride.  Although  this  subjective  overall  checkride 
gra^e  would  appear  to  be  a  good  measure  of  T-41  PIP  perfor¬ 
mance,  it  was  not  available  in  the  sample  of  data  collected, 
and  therefore  cannot  be  used  in  this  study. 
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Chapter  Three 


PILOT  TRAINING  PERFORMANCE  MEASURES 

A  great  deal  of  data  is  maintained  on  each  student  through¬ 
out  pilot  training.  However,  after  graduation,  only  a  limited 
amount  of  data  is  maintained  by  Headquarters  Air  Training 
Command  (HQ  ATC)  in  its  automated  data  base  (17: — ).  This 
chapter  will  describe  the  performance  measures  and  other  data 
maintained  by  HQ  ATC  and  identify  those  items  that  will  be  used 
in  this  study. 


CATEGORY  CHECKS 


UPT  students  are  trained  and  evaluated  in  the  areas  of 
flying  training,  ground  training,  and  officer  development 
throughout  the  course  (7:1).  Evaluation  of  a  student's  flying 
performance  includes  six  major  checkrides,  three  during  the  T- 
37  phase  of  instruction  and  three  during  the  T-38  phase.  These 
checkrides  are  identified  by  category  as  follows  (7:23,24,27, 
40,46,50) : 


T-37  T-38 

Mid  Phase  Evaluation  Day  Contact  Check 

Final  Contact  Evaluation  Navigation/Instrument  Check 

Aircraft  Instrument  Evaluation  Day  Two-Ship  Formation  Check 

On  each  UPT  checkride,  the  student's  characteristic  perfor¬ 
mance  on  each  of  a  prescribed  group  of  maneuvers  is  evaluated. 
The  grade  awarded  on  any  maneuver  is  determined  by  the  same 
absolute  grading  scale  (U/F/G/E)  used  in  the  T-41  PIP  program 
described  in  Chapter  Two  of  this  study  (7:11). 

After  rating  the  student's  performance  in  the  individual 
maneuvers,  the  evaluator  awards  an  overall  grade  of  Excellent 
(E) ,  Good  (G) ,  or  Unsatisfactory  (U)  for  the  checkride,  based 
on  a  subjective  evaluation  of  the  student's  overall  perfor¬ 
mance.  However,  if  any  individual  maneuver  on  a  checkride  is 
graded  "U,"  the  overall  grade  for  the  checkride  is  Unsatisfac¬ 
tory.  A  checkride  will  also  be  rated  Unsatisfactory  if  any 
maneuver  is  graded  below  the  minimum  grade  required  by  the 
syllabus  for  that  maneuver  (7:11).  For  example,  if  a  maneuver 
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requires  a  minimum  grade  of  "G,"  a  student  scoring  an  "F"  on 
that  maneuver  would  receive  an  overall  checkride  grade  of  "U." 

It  is  also  possible  for  a  student  to  receive  an  overall 
checkride  grade  of  *N"  {Not  Graded)  instead  of  "U,"  "G,"  or 
"E."  Under  certain  circumstances,  when  a  student's  flying 
progress  is  determined  to  be  substandard  during  training,  the 
student  will  be  required  to  fly  a  progress  checkride  to  eval¬ 
uate  the  student's  weak  ar^as  of  performance.  If  the  student 
successfully  pusses  the  progress  check,  and  the  following  sor¬ 
tie  in  the  syllabus  is  one  of  the  six  category  checks,  at  the 
IP's  discretion  the  category  check  need  not  be  flown  provided 
that  all  the  category  check  objectives  were  satisfied  on  the 
progress  check.  In  this  situation,  the  category  check  would 
receive  an  overall  score  of  "N"  (Not  Graded)  (7:11-14). 


CATEGORY  CHECK  COMPOSITE 


In  addition  to  the  individual  category  checkride  scores,  HQ 
ATC/DOTC  (Curriculum  Division,  Flight  Operations  Directorate) 
computes  an  unofficial  composite  checkride  score  for  each  UPT 
graduate.  This  score  is  designed  to  be  a  representative  indi¬ 
cator  of  overall  checkride  performance  that  can  be  used  to 
differentiate  the  higher  performing  students  from  the  lower. 

The  score  is  computed  by  taking  each  category  checkride  score 
(U/N/G/E)  and  awarding  a  point  value  for  that  score  according 
to  the  following  scale  (17:  —  ): 


Score  Points 

U  (Unsatisfactory)  0 

N  (Not  Graded)  0 

G  (Good)  2 

E  (Excellent)  3 


The  category  check  composite  icore  is  then  computed  by 
adding  the  six  category  checkrid;:  point  values  together. 

Because  each  category  checkride  score  represents  the  lowest 
checkride  score  awarded  the  student  in  that  category,  a  "U"  is 
recorded  if  the  student  initially  fails  the  checkride  even 
though  the  student  subsequently  passes  the  recheck  for  that 
category.  By  awarding  zero  points  for  initial  failure  on  any 
category  checkride,  a  substantial  point  differential  between 
students  with  no  failures  and  those  with  one  or  more  failures 
can  be  established  (17: — ).  In  the  case  of  an  "N"  (Not  Graded) 
score,  the  student's  substandard  training  performance  prompted 
a  progress  check  that  was  accomplished  in  lieu  of  the  regular 
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category  check  (7:13-14).  Therefore,  the  nNH  score  is  treated 
like  a  "U"  in  computing  the  checkride  composite,  and  zero 
points  are  awarded  for  that  category  check.  It  should  be  noted 
that  this  composite  checkride  score  exists  only  for  the 
internal  use  of  HQ  ATC/DOTC  in  helping  to  distinguish  relative 
performance  between  UPT  students  (17: — ). 


ATRB  RECOMMENDATION 

Sach  ATC  flying  training  wing  convenes  an  Advanced  Training 
Recommendation  Board  (ATRB)  to  evaluate  each  USAF  student's 
capability  and  potential  to  assume  the  varying  mission  require¬ 
ments  of  Air  Force  aircraft  weapons  systems  immediately  upon 
graduation  (6:1)  The  ATRB  procedures  outlined  below  apply  to 
USAF  students  in  both  UPT  and  ENJJPT  (6:2). 

Prior  to  the  convening  of  the  ATRB ,  supervisors  of  each 
flying  training  squadron  meet  to  review  the  characteristic 
performance  of  each  student  and  make  tentative  recommendations 
for  advanced  training.  Among  the  factors  considered  in  this 
evaluation  are  the  student's  performance  in  flying,  academics, 
emergency  procedures,  remedial  training,  and  individual  IP 
recommendations  (6:1) . 

The  ATRB  meets  near  the  end  of  the  pilot  training  course  to 
make  final  advanced  training  recommendations  on  each  student. 

It  reviews  the  advanced  training  recommendations  forwarded  by 
the  flying  training  squadrons  and  carefully  considers  each 
student's  characteristic  performance,  demonstrated  capability, 
and  potential  to  perform  duties  in  the  various  Air  Force  air¬ 
craft  upon  graduation.  The  board  first  identifies  those  male 
students  to  be  recommended  for  fighter,  attack,  and  reconnais¬ 
sance  (FAR)  assignments.  It  then  determines  which  of  these  FAR 
students  should  also  be  recommmended  for  ATC  IP  duty.  From  the 
students  not  recommended  for  FAR  duties,  the  board  selects 
additional  students  for  ATC  IP  recommendation.  Females  and 
USAFA  graduates  with  TTB  ONLY  (tanker,  transport,  and  bomber) 
vision  waivers  may  be  included  in  this  IP  group.  The  remaining 
students  in  the  class  are  then  eligible  for  TTB  assignments 
only  (6 : 1-2) . 


DISTINGUISHED  GRADUATE 

Each  UPT/ENJJPT  ilying  training  wing  selects  students  from 
each  claso  as  distinguished  graduates.  In  order  to  qualify  as 
a  distinguished  graduate  (DG) ,  a  student  must  have  demonstrated 
outstanding  officer  qualities  and  leadership  ability  and  be  in 
the  top  10  percent  of  the  graduating  class.  The  following  fac¬ 
tors  are  used  in  evaluating  each  student  (5:5): 
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a)  Flying  training  record 

b)  Officer  qualities 

c)  Academic  perf ornMiice 

d)  Attitude,  capability,  and  rated  potential 

e)  Student’s  background  and  personnel  records 


ATC  COMMANDER'S  TROPHY 

Each  UPT/ENJJPT  flying  training  wing  is  authorized  to  award 
the  ATC  Commander's  Trophy  to  the  most  outstanding  graduate  in 
each  pilot  training  class  (5:5).  Students  receiving  this  award 
are  in  the  top  10  percent  of  their  class  and  are  thus  also  DGs 
of  their  class.  Therefore,  the  evaluation  criteria  used  in 
this  determination  are  the  same  as  those  listed  above  for  the 
DG  award  (17 :  — )  . 


OTHER  DATA 

The  following  performance  measures  and  data  are  maintained 
by  HQ  ATC/DOTC  (17: — )  but  are  not  used  for  comparison  in  this 
study: 

a)  Number  of  flying  hours  —  the  number  of  hours  flown  in 
the  T-37  and  T-38  phases  of  training,  as  well  as  the  combined 
total.  In  the  student  sample  used  for  this  study,  total  flying 
hours  ranged  from  a  low  of  169.3  to  a  high  of  202.3,  with  most 
values  falling  between  175  and  190.  Because  of  the  limited 
variation  in  the  number  of  hours  flown  to  complete  the  program, 
this  data  was  not  included  among  the  performance  factors  eval¬ 
uated  in  this  study. 

b)  Aircraft  assignment  —  the  type  of  aircraft  to  which  the 
student  was  actually  assigned  in  the  first  assignment  following 
graduation.  The  aircraft  assignment  can  vary  from  the  ATRB 
recommendation  in  the  cases  of  the  IP  and  FAR  recommendations. 
These  recommendations  are  necessary  but  not  sufficient  condi¬ 
tions  for  assignment  to  IP  and  FAR  duties.  Students  without 
either  one  of  these  recommendations  are  assigned  to  TTB  air¬ 
craft.  Students  with  only  the  IP  recommendation  will  be 
assigned  to  either  IP  or  TTB  duties.  Those  with  only  the  FAR 
recommendation  can  be  assigned  to  either  FAR  or  TTB  aircraft, 
while  those  with  both  the  FAR  and  IP  recommendations  can  be 
assigned  to  IP,  FAR,  or  TTB  duties  (17: — ).  Because  of  this 
diversity  in  actual  aircraft  assignments,  this  study  will  con- 
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aider  the  ATRB  recommendation  to  be  a  better  measure  of  overall 
pilot  training  performance  than  the  actual  aircraft  assignment. 

c)  Class  number,  training  base,  and  graduation  date  — 
self-explanatory.  For  the  purposes  of  this  study,  all  UPT 
Oases  and  classes  will  be  considered  to  be  uniform  with  respect 
to  training  and  student  performance. 


PERFORMANCE  MEASURES  SELECTED 


UPT 


This  study  will  use  the  UPT  category  check  composite,  the 
ATRB  recommendation,  the  DG  award,  and  the  ATC  Commander's 
Trophy  award  in  comparing  pilot  training  performance  with  T-41 
PIP  performance.  The  category  check  composite  allows  compari¬ 
son  of  overall  student  checkride  performance  spanning  the 
entire  training  program.  The  ATRB  recommendation  allows  an 
overall  assessment  of  student  performance  based  on  a  final 
recommendation  for  aircraft  duties,  with  FAR  and  IP  recom¬ 
mendations  considered  to  indicate  higher  overall  performance 
than  TTB  recommendations.  The  DG  and  ATC  Commander's  Trophy 
awards  identify  a  small  percentage  of  UPT  graduates  at  the  very 
top  of  their  class  and  are  useful  in  identifying  the  students 
whose  performance  has  been  the  best  overall. 

ENJJPT 


Very  little  performance  data  on  ENJJPT  students  was  avail¬ 
able  for  this  study.  Although  ENJJPT,  like  UPT,  gives  ATRB 
recommendations,  designates  students  as  DGs  and  awards  the  ATC 
Commander’s  Trophy,  this  data  is  not  maintained  by  HQ  ATC/DOTC. 
Available  data  was  limited  to  class  number,  graduation  date, 
number  of  flying  hours,  aircraft  assignment,  and  individual 
category  checkride  scores  (17: — ). 

The  first  three  of  these  available  items  were  rejected  for 
the  same  reasons  as  in  UPT.  The  fourth  item,  aircraft  assign¬ 
ment,  was  not  used  because  all  but  three  of  the  69  graduates 
were  assigned  to  FAR  or  IP  duties,  therefore  offering  minimal 
opportunity  to  differentiate  student  performance.  The  last 
item,  individual  checkride  scores,  was  not  used  for  several 
reasons.  ENJJPT  checkrides  are  graded  on  a  pass/fail  basis 
rather  than  on  the  U/N/G/E  scale  used  in  UPT  (17: — ).  Addi¬ 
tionally,  the  number  of  ENJJPT  checkrides  varied  within  the 
sample,  with  earlier  students  (USAFA  class  of  1984)  completing 
eight  checkrides  and  later  students  (USAFA  class  of  1985) 
completing  10.  Also,  there  was  minimal  variation  in  the  number 
of  checkrides  initially  failed,  again  offering  minimal  oppor¬ 
tunity  to  differentiate  student  performance. 
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Therefore,  because  of  the  unavailability  of  other  meaning¬ 
ful  performance  measures,  this  study  will  use  graduation/alimi- 
nation  as  the  only  measure  of  BNJJPT  performance  to  be  investi¬ 
gated.  Because  of  the  demanding  nature  of  the  program,  ENJJPT 
graduation  will  be  considered  to  be  a  strong  indicator  of 
success  in  pilot  training. 
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Chapter  Pour 


DATA  ANALYSIS 


DATA  GATHERING 

Tha  data  used  in  this  study  was  eollactad  from  two  sources: 
USAFA  and  HQ  ATC.  USAFA  provided  records  of  cadet  performance 
in  its  T-41  PIP  program  for  cadets  in  the  graduating  classes  of 
1984  and  1985,  which  were  the  only  classes  for  which  data  had 
been  compiled.  It  also  provided  a  listing  of  the  Social 
Security  Numbers  (SSNs)  of  all  graduates  of  these  two  USAFA 
classes.  HQ  ATC/DOTC  provided  records  of  the  pilot  training 
classes  (both  UPT  and  ENJJPT)  in  which  graduates  of  the  USAFA 
classes  of  1984  and  1985  had  been  enrolled,  and  also  a  list  of 
eliminees  from  those  pilot  training  classes.  The  data  records 
from  both  sources  were  entered  into  files  on  the  Air  University 
Honeywell  6000  computer  system  for  file  manipulation  and  data 
processing. 

In  order  to  compare  the  T-41  PIP  performance  with  the  pilot 
training  performance  of  USAFA  graduates  of  1984  and  1985,  the 
T-41  PIP  performance  data  of  each  graduate  needed  to  be  matched 
by  SSN  with  the  corresponding  pilot  training  performance  data. 
The  first  step  in  this  process  was  to  compare  the  SSNs  of  the 
individual  UPT  and  ENJJPT  performance  records  with  those  con¬ 
tained  in  the  USAFA  graduate  SSN  file.  This  process  allowed 
the  identification  and  collection  of  those  UPT/ENJJPT  graduates 
who  were  also  USAFA  graduates  of  the  classes  of  1984  and  1985. 
Next,  these  records  were  matched  by  SSN  with  the  T-41  perfor¬ 
mance  records  of  these  two  classes,  and  the  corresponding  T-41 
data  was  merged  into  the  pilot  training  record  of  each 
UPT/ENJJPT  graduate.  Because  of  the  differences  in  record 
structure  and  content,  the  UPT  and  ENJJPT  records  were  main¬ 
tained  in  separate  files. 

In  addition  to  merging  the  T-41  PIP  and  pilot  training 
performance  data  for  UPT/ENJJPT  graduates,  the  T-41  PIP 
performance  data  for  UPT/ENJJPT  eliminees  was  separated  into  a 
separate  file  for  analysis.  Although  no  pilot  training  perfor¬ 
mance  data  was  available  on  these  eliminees,  the  reason  for 
pilot  training  elimination  was  available  for  study. 
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As  a  result  of  these  matching  and  merging  operations,  a 
sample  of  1007  graduates  of  USAFA  classes  1984  and  1985  was 
created.  This  sample  consisted  of  762  UPT  graduates,  68  ENJJPT 
graduates,  and  177  UPT/ENJJPT  eliminees.  However,  there  are 
some  limitations  to  this  sample  that  deserve  mention. 

Not  all  of  the  pilot  training  performance  data  for  the 
USAFA  class  of  1985  was  available  for  this  study.  ATC  provided 
performance  data  only  through  UPT  class  8701  and  ENJJPT  class 
8608.  Therefore,  pilot  training  data  from  approximately  15 
percent  of  the  USAFA  class  of  1985  was  unavailable  due  to 
graduation  dates  later  than  the  collection  date  for  the  data  to 
be  used  in  this  study.  This  percentage  was  computed  based  cn 
the  number  of  class  of  1985  pilot  training  records  collected 
versus  the  number  of  cadets  whose  first  active  duty  assignment 
was  UPT  or  ENJJPT  (18:  —  ). 

Another  limitation  exists  in  the  T-41  PIP  performance  data. 
The  T-41  Checkride  Score  was  unavailable  for  125  of  the  1007 
USAFA  graduates  in  the  sample.  The  data  for  this  performance 
measure  was  completely  missing  for  one  of  the  five  T-41 
training  sessions  conducted  during  the  1983-84  school  year  for 
the  class  of  1984. 


DATA  ANALYSIS  TOOLS 

After  the  T-41  PIP  and  pilot  training  data  had  been  matched 
and  merged,  it  was  ready  for  analysis.  The  Statistical  Package 
for  the  Social  Sciences  (SPSS)  Information  Analysis  System  was 
used  for  this  purpose.  SPSS  is  a  comprehensive,  integrated 
computer  software  package  for  managing,  analyzing,  and 
displaying  information  (2:v) .  Available  on  the  Air  University 
Honeywell  6000  computer  system,  it  allowed  a  variety  of  statis¬ 
tical  analysis  techniques  to  be  used  in  analyzing  the  rela¬ 
tionships  between  T-41  PIP  and  pilot  training  performance. 


ASSUMPTIONS 

In  analyzing  the  pilot  training  performance  data,  this 
study  will  make  the  following  assumptions: 

1)  Higher  UPT  checkride  composite  scores  indicate  better 
UPT  overall  performance  than  do  lower  scores.  Although  the 
computation  of  this  composite  score  is  arbitrary,  it  is  assumed 
to  reflect  with  reasonable  accuracy  the  quality  of  the  stu¬ 
dent’s  overall  performance  in  UPT. 

2)  FAR  and  FAR/IP  recommendations  by  the  ATRB  indicate 
better  overall  UPT  performance  than  TTB  or  TTB/IP  recommenda- 
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tions.  This  is  consistent  with  the  treatment  of  these  recom¬ 
mendations  by  ATC  (17: — ).  Therefore,  for  the  purpose  of 
simplicity,  this  study  will  combine  FAR  and  FAR/IP  recommen¬ 
dations  into  a  single  "FAR"  recommendation,  and  will  in  a 
similar  manner  combine  TTB  and  TTB/IP  into  a  single  "TTB" 
recommendation . 

3)  As  pointed  out  in  the  DATA  GATHERING  section  above,  not 
all  pilot  training  performance  data  is  available  for  the  USAFA 
class  of  1985  due  to  the  cutoff  date  for  data  collection.  It 
is  assumed  that  the  available  data,  representing  approximately 
85  percent  of  the  pilot  training  students  in  this  class,  is 
representative  of  the  class  as  a  whole. 

4)  As  was  also  discussed  above,  the  T-41  Checkride  Score 
was  unavailable  for  125  of  the  graduates  in  this  sample.  This 
missing  data  represented  one  of  the  five  T-41  training  sessions 
conducted  during  the  1983-84  school  year.  Therefore,  it  is 
assumed  that  the  scores  from  this  missing  session  would  be 
distributed  similarly  to  the  other  nine  sessions  making  up  the 
sample,  and  that  the  available  data  is  representative  of  the 
sample  as  a  whole. 

5)  ENJJPT  students  will  be  assumed  to  be  uniform  in  their 
pilot  training  performance,  with  the  exception  of 
graduation/elimination  from  the  program.  As  discussed  in 
Chapter  Three,  the  pilot  training  performance  measures  avail¬ 
able  for  UPT  graduates  were  not  available  for  the  ENJJPT  grad¬ 
uates.  In  addition,  there  was  little  variation  between  the 
ENJJPT  students  in  their  T-41  PIP  performance.  All  69  ENJJPT 
students  had  received  a  "5”  Prediction  for  UPT  rating,  with  the 
exception  of  a  single  "4."  Therefore,  ENJJPT  students  will  be 
included  only  in  the  graduation/elimination  analysis  in  this 
study. 

6)  No  distinction  is  made  between  male  and  female  grad¬ 
uates  in  the  sample.  Although  females  do  not  receive  FAR 
recommendations  in  pilot  training,  the  available  data  did  not 
identify  graduates  by  sex.  Although  this  has  some  effect  on 
the  distribution  between  FAR  and  TTB  assignments  in  the  sample, 
the  effect  is  assumed  to  be  minimal  due  to  the  relatively  small 
number  of  female  student  pilots  as  compared  to  male.  There¬ 
fore,  the  sex  of  the  graduates  in  this  sample  will  not  be 
considered  in  analyzing  and  reporting  the  data. 


DATA  CHARACTERISTICS 

Before  performing  statistical  analysis  on  research  data,  it 
is  important  to  identify  the  characteristics  of  the  data  ele¬ 
ments  being  studied.  Different  types  of  statistical  tests  are 
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appropriate  only  Cor  certain  types  of  data,  and  can  produce 
results  that  are  erroneous  or  not  meaningful  when  used  with 
inappropriate  types  of  data.  The  data  types  used  in  this  study 
ara  reviewed  below  (1:4-8;  13:18-20): 


T-41  Performance  Data 
Nrme 

T-41  Checkride  Score 
Prediction  for  UPT 


Range  of  Values 
0.0  -  100.0 
1,  2,  3,  4,  5 


Data  Type 

continuous 

categorical 


UPT  Performance  Data 
Name 

UPT  Checkride  Composite 
ATRB  Recommendation 
Distinguished  Graduate 
ATC  Commander ' s  Trophy 


Range  of  Values 
0,1,2, .  .  .  18 
FAR,  TT3* 
YES,  NO 
YES,  NO 


Data  Type 

categorical 

dichotomous 

dichotomous 

dichotomous 


*  NOTE:  FAR  and  FAR/IP  have  been  combined, 
as  have  TTB  and  TTB/IP. 

Three  of  the  six  data  items  are  nonnumeric,  and  only  one  of 
the  three  remaining  data  items  is  considered  to  be  continuous. 
In  addition,  these  data  items  measure  subjective  evaluations  of 
performance  rather  than  hard,  precise  physical  data.  For  these 
reasons,  this  study  will  concentrate  on  identifying  general 
trends  and  relationships  between  the  data  items,  rather  than 
focusing  on  rigorous  statistical  analysis  of  precise  functional 
relationships  between  these  items.  However,  some  basic  tests 
for  statistical  significance  and  correlation  between  perfor¬ 
mance  measures  will  be  performed  also. 


ANALYSIS 

T-41  Prediction  for  UPT  versus  UPT  Performance 

The  relationship  between  the  T-41  Prediction  for  UPT  score 
and  UPT  performance  is  shown  in  Tables  1  through  3.  For  this 
analysis,  the  sample  was  divided  into  groups  based  on  the  value 
of  the  Prediction  for  UPT  score.  It  should  be  noted  that  there 
were  no  UPT  graduates  in  the  entire  sample  who  received  a  "1" 


17 


Prediction  for  UPT  score.  For  each  group  ("2"  through  "5"), 

±he  average  (mean,  median,  and  mode)  values  of  the  UPT  check- 
ride  composite  are  listed.  Also,  the  percentages  of  students 
awarded  FAR  and  TTB  recommendations,  and  the  percentages  desig¬ 
nated  as  ATC  Commander's  Trophy  winners  CATC")  and  DGs,  are 
listed  for  each  group. 

This  analysis  indicates,  in  general,  a  direct  relationship 
between  the  T-41  Prediction  for  UPT  rating  and  overall  perfor¬ 
mance  in  UPT.  In  general,  all  UPT  performance  measures  de¬ 
crease  as  the  value  of  the  Prediction  for  UPT  score  decreases. 
The  exceptions  to  this  trend  are  the  mode  UPT  Checkride  Com¬ 
posite  score  for  both  classes  combined  and  for  the  class  of 
1985,  and  the  percentage  of  ATC  Commander's  Trophy  winners  and 
DGs  in  the  class  of  1985  for  the  group  scoring  a  "5"  in  the 
Prediction  for  UPT  rating. 

Upon  examination  of  the  distribution  of  the  UPT  Checkride 
Composite  scores,  it  appears  that  the  mode  is  not  a  good  indi¬ 
cator  of  "average"  performance  in  this  area.  For  example,  in 
Table  1,  the  mode  UPT  Checkride  Composite  score  in  the  "5" 
Prediction  for  UPT  group  for  both  classes  combined  is  11,  which 
is  lower  than  the  mode  for  the  "4”  and  "3"  groups.  However,  if 
the  source  data  for  that  "5"  group  were  to  be  examined,  it 
would  show  that  nine  students  scored  the  mode  of  11,  36  stu¬ 
dents  scored  higher  than  the  mode,  while  only  13  students 
scored  lower.  Because  this  mode  is  significantly  displaced 
from  the  "middle"  of  the  group,  and  because  it  deviates  so 
greatly  from  the  mean  and  median  scores,  it  does  not  appear  to 
be  representative  of  "average"  performance  for  the  group.  This 
same  relationship  holds  true  in  Table  3  for  the  groups  scoring 
"5,"  "4,"  and  "3"  in  the  Prediction  for  UPT  category,  where  the 
mode  scores  are  significantly  offset  from  the  mean  and  median 
scores  also. 

Another  apparent  "anomaly"  in  the  general  trend  of  the  data 
occurs  in  Table  3  in  the  percentage  of  ATC  Commander ' '»  Trophy 
winners  and  DGs  in  the  class  of  1985.  In  the  group  scoring  "5" 
in  the  Prediction  for  UPT  category,  there  were  no  ATC  Com¬ 
mander's  Trophy  winners,  and  only  6.7  percent  of  the  group  were 
DGs,  as  compared  to  12.8  percent  for  the  "4"  group. 

A  possible  explanation  for  this  occurrence  can  be  found  in 
examining  the  number  of  students  scoring  a  "5"  in  the  Predic¬ 
tion  for  UPT  category  for  the  class  of  1985,  and  comparing  that 
number  with  the  same  group  in  the  class  of  1984.  In  the  class 
of  1985,  only  44  cadets  were  awarded  a  "5"  in  this  category, 
and  28  of  those  44  cadets  entered  the  ENJJPT  program.  There¬ 
fore,  only  16  cadets  (36  percent)  of  the  "5"  cadets  in  1985 
enrolled  in  UPT.  In  the  class  of  1984,  however,  82  cadets 
scored  a  "5,"  and  only  37  of  those  82  cadets  entered  ENJJPT. 
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Therefor*,  45  cadets  (55  percent)  of  the  "5"  group  entered  UPT. 
Because  the  actual  number  of  cadets  from  1965  in  the  "5"  group 
was  far  less  than  from  the  class  of  1984  (16  versus  45) ,  and 
because  the  percentages  were  lower  also  (36  percent  versus  55 
percent) ,  the  effect  of  even  a  single  student  not  winning  the 
ATC  Commander's  Trophy  or  not  being  designated  a  DG  has  a  large 
impact  on  the  percentages  computed  for  these  awards  in  the  "5" 
group  for  1985. 

Because  the  number  of  student  records  in  both  year  groups 
is  roughly  equivalent  (530  for  1984,  477  for  1985),  it  would  be 
expected  that  the  number  of  "5"  Prediction  for  UPT  scores 
awarded  in  the  1985  group  would  be  roughly  equivalent  or  only 
slightly  less  than  the  number  awarded  in  the  1984  group.  In¬ 
stead,  the  number  is  considerably  less  (44  versus  82).  This 
difference  can  be  partially  accounted  for  by  the  fact  that  the 
1985  group  is  10  percent  smaller  than  the  1984  group.  However, 
the  remainder  of  the  difference  might  be  accounted  for  by 
another  factor. 

Prior  to  1984,  the  number  of  "5"  scores  awarded  in  this 
category  was  unrestricted,  and  each  T-41  IP  awarded  this  score 
based  only  on  his  or  her  subjective  judgment.  For  the  class  of 
1985,  however,  USAFA  changed  the  procedure  for  the  awarding  of 
the  "5"  score.  Although  the  number  of  "5"  scores  awarded 
remained  unrestricted  (no  quota) ,  the  awarding  of  this  score 
was  made  subject  to  a  board  review  to  ensure  that  all  "5" 
ratings  awarded  accurately  reflected  the  student's  potential 
and  performance.  The  net  result  of  this  change  was  to  lower 
the  percentage  of  cadets  receiving  the  "5"  Prediction  for  UPT 
rating  (19: — ) . 

T-41  Checkride  Score  versus  UPT  Performance 


The  relationship  between  the  T-41  Checkride  Score  and  UPT 
performance  is  shown  in  Tables  4  through  6.  For  this  analysis, 
cadets  who  entered  UPT  were  divided  into  five  groups  based  on 
the  menn  and  standard  deviation  of  the  T-41  Checkride  Score 
percentages  in  their  class.  For  both  classes  combined,  the 
mean  and  standard  deviation  were  computed  for  the  entire  sample 
without  regard  to  class.  The  five  groups  represent  an  arbi¬ 
trary  division  into  artificial  "A,"  "B,"  "C,"  "D,"  AND  "F" 
groups  based  on  the  scale  shown  below: 


A 


score  >  {mean  +  1  std  dev) 


B  mean  <  score  <*  (mean  +  1  std  dev) 

C  (mean  -  1  std  dev)  <  score  <=  mean 

D  (mean  -  2  std  dev)  <  score  <=  (mean  -  1  std  dev) 

F  score  <=  (mean  -  2  std  dev) 


This  division  of  scores  into  artificial  "grade"  groups  is 
consistent  with  the  award  of  letter  grades  in  many  academic 
courses  at  USAFA.  Although  the  T-41  PIP  program  remains  a 
pass/fail  course,  these  divisions  were  made  in  order  to  divide 
the  cadets  into  relative  groups  in  order  to  compare  the  overall 
UPT  performance  of  each  group  with  other  groups.  The  UPT 
performance  data  is  listed  in  the  same  manner  as  it  was  for  the 
comparison  of  the  T-41  Prediction  for  UPT  score  with  perfor¬ 
mance  in  UPT  in  Tables  1  through  3. 

This  analysis  indicates,  in  general,  a  direct  relationship 
between  the  T-41  Checkride  Score  and  overall  performance  in 
UPT.  Generally,  all  UPT  performance  measures  decrease  as  the 
"grade"  in  the  T-41  Checkride  Score  decreases.  Minor  differen¬ 
ces  from  this  general  trend  occur  in  the  FAR/TTB  distribution 
in  the  "D"  group  for  the  class  of  1984  and  in  the  "C"  group  for 
the  class  of  1985.  Other  minor  differences  appear  in  the  mean, 
median,  and  mode  of  the  UPT  Checkride  Composite  scores  in  the 
"D"  and  "F"  groups. 

The  most  notable  deviations  from  this  trend,  however,  occur 
in  the  class  of  1985  in  the  ATC  Commander's  Trophy  in  the  "A" 
group  and  the  DG  category  in  the  "A"  and  "F"  groups.  It  should 
be  noted,  however,  that  if  only  two  of  the  35  "A"  students  in 
1985  had  won  the  ATC  Commander's  Trophy  (instead  of  0),  the 
percentage  of  trophy  winners  would  be  the  same  as  for  1984. 

But  it  would  take  a  total  of  eight  students  (instead  of  the 
actual  three)  being  designated  DG  in  1985  to  equal  the  per¬ 
centage  in  1984.  These  differences  between  1984  and  1935  may 
be  worthy  of  further  study.  However,  since  this  analysis  is 
examining  USAFA  graduate  performance  as  a  whole,  it  will  not: 
focus  extensively  on  differences  between  the  classes. 

The  other  notable  difference  mentioned  was  in  the  DG  per¬ 
centage  in  the  "F"  category  for  the  class  of  1985.  While  no 
students  in  the  "F"  category  were  designated  DG  in  1984,  6.7 
percent  of  the  "F"  students  in  1985  attained  DG  status.  How¬ 
ever,  as  there  are  only  15  students  in  the  "F"  category  in 


1985,  the  6.7  percent  represents  just  one  of  those  15  students 
becoming  a  DG.  Because  of  the  small  number  of  students  in  this 
category,  it  would  appear  that  the  differences  between  the 
classes  o "  1984  and  1985  in  this  area  are  not  significant. 

T-41  Prediction  for  UPT  versus  Graduation  Status 


The  relationship  between  the  T-41  Prediction  for  UPT  score 
and  pilot  training  graduation  status  is  shown  in  Tables  7 
through  9.  For  each  Prediction  for  UPT  group,  the  numbers  and 
percentages  of  that  group  graduating  from  UPT,  graduating  from 
ENJJPT ,  and  being  eliminated  from  either  program  are  shown.  In 
addition,  the  overall  totals  and  percentages  for  all  Prediction 
for  UPT  groups  in  these  three  graduation  status  categories  are 
also  shown. 

An  analysis  of  these  tables  indicates,  in  general,  a  direct 
relationship  between  the  Prediction  for  UPT  score  and  success¬ 
ful  completion  of  UPT.  In  all  three  tables,  as  the  Prediction 
for  UPT  score  decreases,  the  combined  percentage  of  UPT  and 
ENJJPT  graduates  in  that  scoring  group  also  decreases.  Simi¬ 
larly,  these  tables  indicate  a  generally  inverse  relationship 
between  the  Prediction  for  UPT  score  and  elimination  from  pilot 
training.  In  all  three  tables,  as  the  Prediction  for  UPT  score 
decreases,  the  percentage  of  pilot  training  eliminees  in  that 
scoring  group  increases. 

Analysis  of  Eliminees 

Table  10  contains  an  analysis  of  the  T-41  PIP  performance 
of  eliminees  from  both  classes  combined  and  for  each  class 
separately.  It  compares  the  mean  scores  of  the  T-41  Checkride 
Score  and  the  Prediction  for  UPT  score  for  both  UPT  graduates 
and  UPT  eliminees.  (Because  there  was  only  one  eliminee  from 
ENJJPT  in  the  1984  and  1985  year  groups  combined,  the  table 
presents  UPT  data  only) .  The  table  shows  that  for  all  year 
groups,  the  mean  T-41  PIP  performance  scores  for  eliminees  are 
substantially  lower  than  those  for  the  UPT  graduates. 

Table  11  illustrates  the  distribution  of  reasons  for  elimi¬ 
nation  among  the  Prediction  for  UPT  score  groupings  for  the 
classes  of  1984  and  1985  combined.  It  shows  that  over  three- 
fourths  (76.7  percent)  of  the  UPT/ENJJPT  eliminees  were  elimi¬ 
nated  because  of  flying  deficiency.  It  also  shows  that  the 
overwhelming  majority  (150)  of  the  176  eliminees  were  students 
who  scored  either  a  "3"  (95  students)  or  a  "2"  (55  students)  on 
the  T-41  Prediction  for  UPT  rating. 


Analysis  of  Correlation  and  Significance 


Statistical  tests  for  significance  and  correlation  between 
the  T-41  PIP  and  pilot  training  performance  measures  were  also 
conducted  for  both  USAFA  classes  combined.  Table  12  lists  the 
results  of  chi-square  tests  of  statistical  significance  between 
the  pairs  of  performance  measures.  It  shows  that  in  all  cases 
the  computed  value  of  significance  is  less  than  0.01,  indi¬ 
cating  that  there  is  a  statistically  significant  difference 
between  the  expected  and  actual  performance  measure  means 
(13:24-25).  This  difference  indicates  that  any  relationships 
that  exist  between  the  performance  measures  in  the  sample  are 
"real"  and  do  not  occur  randomly  or  by  chance  (13:24). 

Tests  of  statistical  correlation  were  also  performed  for 
the  purpose  of  determining  the  strength  of  the  relationships 
between  the  performance  measures.  Because  the  performance 
measures  compared  are  either  categorical  or  dichotomous  in 
nature  (13:18-20),  a  nonparametric  test  of  correlation,  the 
contingency  coefficient,  was  used  (1:5-7;  13:45,53).  The  re¬ 
sults  of  these  tests  are  presented  in  Table  13.  It  should  be 
noted  that  nonparametric  tests  of  correlation  on  categorical  or 
dichotomous  data  produce  results  that  are  generally  less  accu¬ 
rate  than  parametric  tests  involving  continuous  data  (20: — ). 

As  is  shown  in  Table  13,  the  computed  values  of  correlation  in 
all  cases  indicate  a  weak  to  slightly  moderate  correlation 
(values  less  than  0.4)  between  the  pairs  of  performance 
measures . 

These  tests  for  significance  and  correlation  indicate  that 
statistically  significant  relationships  exist  between  the  T-41 
PIP  and  pilot  training  performance  measures.  However,  they  do 
not  indicate  that  these  measures  are  closely  or  strongly 
related.  Therefore,  these  tests  do  not  appear  to  indicate  as 
strong  a  relationship  between  these  performance  measures  as  is 
indicated  by  the  rest  of  the  analysis  presented  in  this 
chapter. 
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Chapter  Five 


FINDINGS  AND  RECOMMENDATIONS 

FINDINGS 


Based  on  the  preceding  data  analysis,  the  following 
findings  are  presented: 

1)  In  general,  the  T-41  Prediction  for  UPT  score  is 
directly  related  to  performance  in  pilot  training  in  the  sample 
studied.  Higher  Prediction  for  UPT  scores  are  generally  asso¬ 
ciated  with  higher  overall  performance  scores  in  pilot  train¬ 
ing,  and  vice  versa. 

2)  In  general,  the  T-41  Checkride  Score  is  also  directly 
related  to  pilot  training  performance  in  the  sample  studied. 
Higher  T-41  Checkride  Scores  are  generally  associated  with 
higher  overall  pilot  training  performance  scores,  and  vice 
versa. 

3)  The  T-41  Prediction  for  UPT  score  generally  relates 
directly  to  success  in  pilot  training  as  measured  by 
graduation/elimination  status  in  this  sample.  Higher  Predic¬ 
tion  for  UPT  scores  are  generally  associated  with  higher  per¬ 
centages  of  graduation,  and  vice  versa. 

4)  Eliminees  from  pilot  training  in  this  sample  scored 
substantially  lower  in  both  T-41  PIP  performance  measures  than 
did  pilot  training  graduates.  The  majority  of  the  pilot 
training  eliminees  were  eliminated  for  flying  deficiency,  and 
most  eliminees  scored  either  a  ”2"  or  "3"  in  the  T-41 
Prediction  for  UPT  rating. 

5)  Statistical  tests  for  significance  and  correlation 
indicate  that  the  T-41  PIP  and  pilot  training  performance 
measures  are  somewhat  related  in  this  sample.  However,  the 
remainder  of  the  analysis  performed  in  this  study  suggests  that 
stronger  relationships  exist  between  the  performance  measures. 
This  difference  in  results  might  be  partially  explained  by  the 
limited  accuracy  inherent  in  nonparametric  tests  of 
correlation.  In  any  event,  this  difference  is  notable  and  may 
be  of  interest  to  future  studies  in  this  area. 


RECOMMENDATIONS 


The  data  analysis  performed  in  this  study  laads  to  the 
following  recommendations: 

1)  USAFA  should  continue  to  rate  cadets  in  T-41  PIP  using 
the  Prediction  for  UPT  measure  of  performance.  In  addition. 
USAFA  should  consider  adopting  the  T-41  Checkride  Score  per¬ 
centage  as  a  formal  measure  of  cadet  performance,  rather  than 
only  using  it  as  a  quality  control  tool  for  the  IP  force.  The 
relationships  between  these  T-41  performance  measures  and  per¬ 
formance  in  pilot  training  appear  to  be  strong. 

2)  USAFA  should  continue  to  select  cadets  for  ENJJPT  on 
the  basis  of  the  "5N  score  in  the  Prediction  for  UPT  rating. 

In  the  sample  studied,  the  elimination  rate  of  USAFA  T-41  PIP 
graduates  from  ENJJPT  was  negligible,  and  the  overwhelming 
majority  of  these  former  cadets  were  assigned  to  FAR/IP  duties, 
as  expected  by  the  ENJJPT  program. 

3)  USAFA  should  continue  to  award  cadets  a  subjective 
overall  checkride  grade  (U/F/G/E)  in  T-41  PIP.  However,  this 
grade  should  also  be  maintained  on  a  permanent  basis  and  made 
available  for  analysis,  as  should  the  other  two  T-41  PIP 
performance  measures  investigated  in  this  study. 

4)  USAFA  should  further  investigate  the  pilot  training 
performance  of  cadets  who  score  in  the  lower  portions  of  the 
T-41  PIP  performance  measures.  Further  studies  of  these 
graduates  could  indicate  whether  or  not  all  cadets  receiving  a 
"2"  Prediction  for  UPT  rating,  for  example,  should  be  allowed 
to  enter  pilot  training.  Such  studies  might  also  determine 
whether  additional  screening  measures  for  these  cadets  would  be 
warranted  before  accepting  them  for  UPT  entry. 

5)  USAFA  should  begin  an  ongoing  program  to  record  and 
track  historical  data  of  cadet  performance  in  T-41  PIP  and 
graduate  performance  in  the  UPT/ENJJPT  programs.  The  T-41  PIP 
performance  data  used  in  this  study  was  available  only  because 
of  an  ad  hoc  collection  of  this  data  on  a  one-time  basis.  By 
establishing  a  regular  data  collection  system  and  common  data 
base  for  T-41  PIP  and  pilot  training  performance  data,  a  long¬ 
term,  in-depth  study  of  the  relationships  between  these  perfor¬ 
mance  measures  could  be  conducted.  As  a  result  of  such  a 
study,  the  relationship  between  performance  in  T-41  PIP  and 
pilot  training  could  be  better  studied  and  understood.  Thus, 
improvements  in  the  means  of  screening  T-41  PIP  students  for 
UPT  and  predicting  UPT  performance  could  be  developed,  with  the 
goal  of  lowering  pilot  training  attrition  rates. 
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COMPARISON  OP  T-41  PREDICTION  FOR  UPT 


WITH 

UPT  PERFORMANCE 
(BOTH  CLASSES) 


PREDICTION 

FOR  UPT  UPT  CHECK  COMPOSITE 
(MEAN/MEDIAN/MODE) 


ALL 

10. 

.94 

/ 

11 

/’ 

12 

5 

12. 

.90 

/ 

13 

/ 

11 

4 

11. 

.74 

/ 

12 

/ 

12 

3 

10. 

.40 

/ 

11 

/ 

12 

2 

9, 

.37 

/ 

10 

/ 

11 

ATRB 

AWARDS 

FAR 

TTB 

ATC 

DG 

(%) 

(%) 

(%) 

(%) 

52.8 

47.2 

1.2 

7.7 

82.8 

17.2 

5.2 

31.0 

66.0 

34.0 

1.2 

10.9 

45.4 

54.6 

0.8 

3.3 

20.5 

79.5 

0.0 

0.0 

Table  1.  Prediction  for  UPT  versus  UPT  Performance. 
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COMPARISON  OF  T-41  PREDICTION  FOR  UPT 

WITH 

UPT  PERFORMANCE 
(1984) 


PREDICTION 

FOR  UPT  UPT  CHECK  COMPOSITE 

(MEAN/MEDIAN/MODE) 


ALL 

11.02 

/ 

11 

/ 

12 

5 

13.00 

/ 

13 

/ 

15 

4 

11.80 

/ 

12 

/ 

12 

3 

10.43 

/ 

11 

/ 

8 

2 

9.32 

/ 

10 

/ 

8 

ATRB  AWARDS 


FA/. 

TTB 

ATC 

DG 

(%) 

(%) 

(%) 

(%) 

53.8 

46.2 

1.7 

8.9 

81.4 

18.6 

7.0 

39.6 

64.8 

35.2 

1.4 

9.1 

45.9 

54.1 

1.1 

3.8 

26.5 

73.5 

0.0 

0.0 

Table  2.  Prediction  for  UPT  versus  UPT  Performance  (1984) . 
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COMPARISON  OF  T-41  PREDICTION  FOR  UPT 

WITH 

UPT  PERFORMANCE 
(1985) 


PREDICTION 
FOR  UPT 

UPT  CHECK 

COMPOSITE 

FAR 

ATRB 

TTB 

AWARDS 

ATC  DG 

(MEAN/MEDIAN/MODE) 

,  (%) 

(%) 

(%> 

(%) 

ALL 

10.83  / 

11  /  11 

51.3 

48.7 

0.6 

6.2 

5 

12.67  / 

12  /  11 

86.7 

13.3 

0.0 

6.7 

4 

11.67  / 

12  /  14 

67.5 

32.5 

0.9 

12.8 

3 

10.37  / 

11  /  12 

44.5 

55.5 

0.5 

2.7 

2 

9.45  / 

10  /  11 

10.3 

89.7 

0.0 

0.0 

Table  3.  Prediction  for  UPT  versus  UPT  Performance  (1985). 


31 


COMPARISON  OP  T-41  CHECKRIDE  SCORE 

WITH 

UPT  PERFORMANCE 
(BOTH  CLASSES) 


T-41  CHKRIDE 
SCORE  GP 

UPT  CHECK  COMPOSITE 

ATRB 

FAR 

i 

TTB 

AWARDS 
ATC  DG 

( MEAN/MEDIAN /MODE ) 

(%) 

(%> 

(%) 

(%) 

ALL 

10.94 

/ 

11 

/ 

12 

52.8 

47.2 

1.2 

7.7 

A 

12.25 

/ 

12 

/ 

12 

80.9 

19.1 

2.9 

16.1 

B 

11.46 

/ 

12 

/ 

12 

59.7 

40.3 

1.3 

9.9 

C 

10.13 

/ 

10 

/ 

11 

39.2 

60.8 

0.5 

2.8 

D 

9.91 

/ 

10 

/ 

8 

38.6 

61.4 

0.0 

0.0 

F 

9.75 

/ 

10 

/ 

11 

25.0 

75.0 

0.0 

4.2 

Table  4.  T-41  Checkride  Score  versus  UPT  Performance. 
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COMPARISON  OF  T-41  CHECKRIDE  SCORE 

WITH 

UPT  PERFORMANCE 
(1984) 


T-41  CHKRIDE  ATRB  AWARDS 


SCORE  GP 

UPT  CHECK 

COMPOSITE 

FAR 

TTB 

ATC 

DG 

(%) 

(MEAN/MEDIAN/MODE) 

(%) 

(%) 

<%) 

ALL 

11.02 

/ 

11 

/ 

12 

53.8 

46.2 

1.7 

8.9 

A 

12.66 

/ 

13 

/ 

12 

74.3 

25.7 

5.7 

22.8 

B 

11.41 

/ 

12 

/ 

12 

56.2 

43.8 

2.2 

11.7 

C 

10.24 

/ 

11 

/ 

8 

45.5 

54.5 

0.0 

2.0 

D 

9.85 

/ 

10 

/ 

8 

38.5 

61.5 

0.0 

0.0 

F 

9.90 

/ 

11 

/ 

6 

40.0 

60.0 

0.0 

0.0 

Table  5,  T-41  Checkride  Score  versus  UPT  Performance  (1984) . 


33 


COMPARISON  OF  T-41  CHECKRIDE  SCORE 


WITH 

UPT  PERFORMANCE 
(1985) 


T-41  CHKRIDE 

SCORE  GP  UPT  CHECK  COMPOSITE 
(MEAN/MEDIAN/MODE) 


ALL 

10. 

rn 

CO 

/ 

11 

/ 

11 

A 

11. 

,77 

/ 

12 

/ 

14 

B 

11. 

.51 

/ 

12 

/ 

11 

C 

9. 

.99 

/ 

10 

/ 

11 

D 

9. 

.84 

/ 

10 

/ 

8 

F 

9, 

.67 

/ 

9 

/ 

9 

ATRB 

AWARDS 

FAR 

TTB 

ATC 

DG 

(%) 

(%) 

(%) 

(%) 

51.3 

48.7 

0.6 

6.2 

82.9 

17.1 

0.0 

8.6 

62.3 

37.7 

1.3 

9.0 

33.6 

66.4 

0.0 

2.8 

36.8 

63.2 

0.0 

0.0 

20.0 

80.0 

0.0 

6.7 

Table  6.  T-41  Checkride  Score  versus  UPT  Performance  (1985) . 
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Table  7.  Prediction  for  HPT  versus  Graduation  Status 


COMPARISON  OP  T-41  PREDICTION  FOR  UPT 


Table  8.  Prediction  for  UPT  versus  Graduation  Status  (1984) 


COMPARISON  OF  T-41  PERFORMANCE  MEASURES 

WITH 

UPT  GRADUATION  STATUS 


PILOT  TRAINING 
GRAD  STATUS 

MEAN  T-41 
CHECK  SCORE 

MEAN  PREDICTION 
FOR  UPT  SCORE 

GRADUATES 

81.57 

3.39 

BOTH 

CLASSES 

ELIMINEES 

76.36 

2.85 

CLASS  OF 

GRADUATES 

81.84 

3.43 

1984 

ELIMINEES 

75.75 

2.87 

CLASS  OF 

GRADUATES 

81.36 

3.34 

1985 

ELIMINEES 

76.51 

2.84 

Table  10. 


T-41  Performance  versus  UPT  Graduation  Status. 
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DISTK  -BUTION  OP  PILOT  TRAINING  ELIMINEES 

BY 

T-41  PREDICTION  FOR  UPT 
AND 

REASON  FOR  ELIMINATION 
(BOTH  CLASSES) 

PREDICTION  FOR  UPT 


5 

4 

3 

2 

ALL 

FLYING 

3 

12 

71 

49 

135 

(76.7%) 

ACADEMIC 

1 

1 

(0.6%) 

MILITARY 

2 

2 

(1.1%) 

REASON  FOR 

ELIMINATION 

MEDICAL 

6  . 

6 

3 

15 

(8.5%) 

MOA* 

1 

5 

1 

7 

(4.0%) 

*  * 

SIE 

1 

2 

10 

2 

15 

(8.5%) 

OTHER 

1 

1 

(0.6%) 

TOTAL 

4 

22 

95 

55 

176 

(100%) 

*  MANIFESTATION  OF  APPREHENSION 
**  SELF-INITIATED  ELIMINATION 


Table  11.  Distribution  of  Pilot  Training  Eliminees. 
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CHI-SQUARE  TESTS  OF  STATISTICAL  SIGNIFICANCE 

BETWEEN 

T-41  PIP  AND  UPT  PERFORMANCE  MEASURES 
(BOTH  CLASSES) 


PERFORMANCE 

CHI-SQUARE 

DEGREES  OF 

SIGNIFICANCE 

MEASURES 

VALUE 

FREEDOM 

LEVEL 

T-41  PRED  FOR  UPT  vs: 

UPT  CHECKRIDE 
COMPOSITE 

135.6 

64 

0.0000 

ATRB  RECOMM 
(FAR/TTB) 

80.7 

4 

0.0000 

AWARDS 

(ATC/DG) 

66.4 

8 

0.0000 

T-41  CHECKRIDE  SCORE  vs 

UPT  CHECKRIDE 
COMPOSITE 

* 

• 

107.7 

64 

0.0005 

ATRB  RECOMM 
(FAR/TTB) 

53.7 

4 

0.0000 

AWARDS 

(ATC/DG) 

21.3 

8 

0.0064 

TABLE  12.  Chi-Square  Tests  of  Significance. 
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TESTS  OF  STATISTICAL  CORRELATION 
BETWEEN 

T-41  PIP  AND  UPT  PERFORMANCE  MEASURES 
(BOTH  CLASSES) 


PERFORMANCE  CONTINGENCY  COEFFICIENT 

MEASURES  OF  CORRELATION 

T-41  PRED  FOR  UPT  VS: 


UPT  CHECKRIDE 

COMPOSITE 

0.39 

ATRB  RECOMM 

(FAR/TTB) 

0.31 

AWARDS 

(ATC/DG) 

0.28 

T-41  CHECKRIDE  SCORE  GROUP  vs: 

UPT  CHECKRIDE 

COMPOSITE  0.38 

ATRB  RECOMM 

(FAR/TTB)  0.28 

AWARDS 

(ATC/DG)  0.18 


TABLE  13.  Tests  of  Statistical  Correlation. 
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